Papers with auxiliary loss
Augmented Adversarial Trigger Learning (2025.findings-naacl)
Copied to clipboard
| Challenge: | Gradient optimization-based adversarial attack methods can generate jailbreak prompts or leak system prompts. |
| Approach: | They propose an algorithm that enhances negative log-likelihood loss and augments it with auxiliary loss. |
| Outcome: | The proposed approach outperforms current state-of-the-art techniques in nearly 100% of attacks while requiring 80% fewer queries. |
Constraint based Knowledge Base Distillation in End-to-End Task Oriented Dialogs (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for learning task-oriented dialog systems filter irrelevant KB information over a large KB. |
| Approach: | They propose a pairwise similarity filter that respects the n-ary structure in a KB record and an auxiliary loss that helps in separating contextually unrelated KB information. |
| Outcome: | The proposed method outperforms existing state-of-the-art models on three publicly available datasets. |
Supervised Optimism Correction: Be Confident When LLMs Are Sure (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) have demonstrated remarkable success across diverse tasks such as instruction following, code generation, and medical diagnosis. |
| Approach: | They propose a supervised fine-tuning-based auxiliary loss for Q-value estimations during supervised refinement. |
| Outcome: | The proposed method outperforms beam search on GSM8K, MATH, and GAOKAO on reasoning benchmarks. |
Sequence Labeling Parsing by Learning across Representations (P19-1)
Copied to clipboard
| Challenge: | Constituency and dependency parsing are the main abstractions for representing syntactic structure of sentences . constituency parsers are considered disjointed tasks, and their improvements have been obtained separately. |
| Approach: | They propose to add auxiliary loss to constituency parsing paradigms and explore a model that parses both paradigms at no cost. |
| Outcome: | The proposed model outperforms single-task models by 1.05 F1 points and 0.62 UAS points for constituency parsing and dependency parsers. |
Countering the Effects of Lead Bias in News Summarization via Multi-Stage Training and Auxiliary Losses (D19-1)
Copied to clipboard
| Challenge: | Sentence position is a strong feature for news summarization, since the lead often summarizes the key points of the article. |
| Approach: | They propose two techniques to make neural systems sensitive to the importance of content in different parts of the article by using random shuffled sentences to pretrain the model. |
| Outcome: | The proposed techniques improve the performance of a competitive reinforcement learning based extractive system, with the auxiliary loss being more powerful than pretraining. |
FLEXITOKENS: Flexible Tokenization for Evolving Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Widely used subword tokenizers overfragment sequences in unseen domains, languages, and scripts . inefficient tokenizer models can cause overfragments in out-of-distribution domains if not trained properly . |
| Approach: | They propose a byte-level LM with learnable tokenizers to make tokenization adaptive . they propose 'flexitoken' which enables significantly greater flexibility during adaptation . |
| Outcome: | The proposed method significantly reduces token overfragmentation and improves on multilingual benchmarks and domains. |